Latest news

CrowdStrike Incident and Recovery Steps

The recent Crowdstrike incident has caused significant disruptions across the internet, leading to widespread outages. This issue affects windows users worldwide after a CrowdStrike update was pushed, resulting in blue screen errors. The issue occurred due to a defect in a content update for Microsoft users within CrowdStrike.

Manual Recovery Steps

One of our consultants, George Chapman has compiled the following recovery advice based on official guidance from the vendor. If you are dealing with affected systems, the following steps can be taken:

1. Create a Windows PE/RE USB Stick
2. Write and save the following batch file to the USB Stick. Filename is crowdstrike_workaround.bat

a. If BitLocker is not enabled:

@echo off
timeout /t 10
cd C:\Windows\System32\drivers\CrowdStrike
del C-00000291*.sys
wpeutil reboot

b. If BitLocker is enabled:

@echo off
timeout /t 10
manage-bde -unlock c: -recoverypassword [machine specific recovery key]
cd C:\Windows\System32\drivers\CrowdStrike
del C-00000291*.sys
wpeutil reboot

3. Boot to Windows PE/RE Stick – BIOS options may need modifying to allow for USB Boot…
4. Run the following command

cd x:
crowdstrike_workaround.bat

5. Reboot the host normally and this should fix the issue.

Optional: Should a user want to automate running the script, the following should work but isn’t yet tested.

6. Create a startnet.cmd file in the \Windows\System32 directory on the Windows PE image:

batchCopy codewpeinit
X:\crowdstrike_workaround.bat

Automated workaround in safe mode using Group policy

A cyber threat intelligence analyst named Arda Büyükkaya has managed to create an automated workaround in safe mode using group policies. (Automated CrowdStrike BSOD Workaround in Safe Mode using Group Policy · GitHub)

Here’s how you can implement this solution:

1. Create the PowerShell Script

Create a powershell script that deletes the problematic CrowdStrike driver file causing the blue screens and handles the safemode boot and revert

Below is the powershell script:

# CrowdStrikeFix.ps1
# This script deletes the problematic CrowdStrike driver file causing BSODs and reverts Safe Mode

$filePath = "C:\Windows\System32\drivers\C-00000291*.sys"
$files = Get-ChildItem -Path $filePath -ErrorAction SilentlyContinue

foreach ($file in $files) {
    try {
        Remove-Item -Path $file.FullName -Force
        Write-Output "Deleted: $($file.FullName)"
    } catch {
        Write-Output "Failed to delete: $($file.FullName)"
    }
}

# Revert Safe Mode Boot after Fix
bcdedit /deletevalue {current} safeboot

2. Create a GPO for Safe Mode

• Open the Group Policy Management Console (GPMC).
• Right-click on the appropriate Organizational Unit (OU) and select Create a GPO in this domain, and Link it here….
• Name the GPO, for example, “CrowdStrike Fix Safe Mode”.

3.Edit the GPO

• Right-click the new GPO and select Edit.
• Navigate to Computer Configuration -> Policies -> Windows Settings -> Scripts (Startup/Shutdown).
• Double-click Startup, then click Add.
• In the Script Name field, browse to the location where you saved CrowdStrikeFix.ps1 and select it.
• Click OK to close all dialog boxes.

4.Force Safe Mode Boot Using a Script

Create another PowerShell script to force Safe Mode boot and link it to a GPO for immediate application, below is the powershell script to do this:

# ForceSafeMode.ps1
# This script forces the computer to boot into Safe Mode

bcdedit /set {current} safeboot minimal
Restart-Computer

5.Create a GPO to Apply the Safe Mode Script

• Open the Group Policy Management Console (GPMC).
• Right-click on the appropriate Organizational Unit (OU) and select Create a GPO in this domain, and Link it here….
• Name the GPO, for example, “Force Safe Mode”.
• Right-click the new GPO and select Edit.
• Navigate to Computer Configuration -> Policies -> Windows Settings -> Scripts (Startup/Shutdown).
• Double-click Startup, then click Add.
• In the Script Name field, browse to the location where you saved ForceSafeMode.ps1 and select it.
• Click OK to close all dialog boxes.

6.Apply the GPOs

• Make sure the Force Safe Mode GPO is applied to the affected computers first.
• The computer will boot into Safe Mode and execute the CrowdStrikeFix.ps1 script.
• Once the issue is fixed, the script will revert the boot settings to normal mode.

These instructions should help mitigate the impact and restore operations.

Conclusion

In conclusion, today’s CrowdStrike outage has caused a significant amount of disruption, throwing IT teams and business teams into a state of emergency. By following the recovery steps provided we hope that systems can be restored swiftly, and normal operations can be resumed as soon as possible.